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An  important  question  in  the  stability  and  control  of 
stochastic  systems  is  the  determination  of  the  limiting  long 
time  behavior  of  the  system  using  a fixed  control.  This  paper 
answers  that  question  in  the  case  of  a stochastic  system  per- 
turbed by  a small  additive  noise  term  where  the  control  is 
such  that  the  corresponding  deterministic  system  has  a stable 
limit  cycle. 

It  is  shown  that  in  the  limit  of  large  time  the  stochastic 
system  is  near  the  limit  cycle.  This  is  a stability  result. 
Moreover,  one  can  compute  approximately  at  which  portion  of  the 
limit  cycle  one  is  most  likely  to  be  found.  Further  various 
stationary  averages  can  be  computed. 

These  results  will  be  of  use  in  designing  approximate  con- 
trols for  stationary  stochastic  control  systems.  As  a by- 
product, the  above  work  allows  one  to  deduce  some  additional 
results  on  singular  perturbation  problems  in  partial  differen- 
tial equations. 


This  research  was  supported  in  part  by  AFCSR  Grant  77-3796. 
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Stochastically  Perturbed  Limit  Cycles 
Charles  J.  Holland 

1.  Introduction.  In  this  paper  we  examine  the  effe 
perturbing  certain  deterministic  dynamical  systems  possessing 
a stable  limit  cycle  by  an  additive  white  noise  term  with  small 
intensity.  We  place  assumptions  on  the  system  guaranteeing 
that  when  noise  is  present  the  corresponding  random  process 
generates  an  ergodic  probability  measure.  We  then 
determine  the  behavior  of  the  invariant  measure  when  the  noise 
intensity  is  small.  This  can  be  considered  as  a purely  probabi- 
listic result.  Using  this  probabilistic  result  we  are  also  able 
to  determine  the  limiting  large  time  behavior  of  certain  singu- 
larly perturbed  second  order  partial  differential  equations 
associated  with  the  random  process  through  It^'s  rule. 

For  a model  of  our  probabilistic  process  we  consider  the 
ItS  stochastic  differential  equation 

d£  = f($)dt+  (2e)l/2a(Odw(t)  . (l) 

In  (1)  f is  an  n vector,  c is  an  nxk  matrix,  w is  k dimensional  . 
Brownian  motion  and  e is  our  small  positive  parameter.  For  e = 0 
we  have  the  corresponding  deterministic  dynamical  system.  For 
e _>  0 denote  by  l-^(t)  the  solution  to  (l)  with  initial  condition 
C(0)  = x.  Then  throughout  the  following  assumptions  are  made. 
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(Al)  f.,  a.  . are  of  class  C2(Rn). 

* 

(A2)  For  e = 0 the  system  has  a unique  limit  cycle  denoted 
by  T. 

(A3)  There  exists  at  most  a finite  number  of  critical  points 

(places  x*  where  f(x*)  = 0).  At  each  critical  point  the 

matrix  f (x*)  has  only  eigenvalues  with  positive  real 

* T 

parts.  At  each  critical  point  the  matrix  a(x  )a  (x*)  is 
positive  definite. 

(A4)  For  any  compact  set  B not  containing  critical  points  and 
any  6 > 0 there  exists  T > 0 (depending  upon  B,  6)  such 
that  if  x e B,  then 

d(e°(t),D  < 6 for  t _>  T . 

Here  d denotes  the  Euclidean  distance  function. 

(A5)  There  exists  > 0 such  that  for  0 < e < eQ  the  stochastic 
differential  equation  possesses  an  ergodic  measure 

with  a density.  For  any  6 > 0 there  exists  R > 0 such 
that 

P.e(B(R)C]  < 6 for  0 < e < Eq 

where  B(R)  = (x:  |x|  R). 

Then  we  prove  the  following  result. 

Theorem  1.  (Cl).  If  B is  any  open  set  such  that  r c B,  then 


lim  hE(B)  = 1 . 

ejo 


(C2).  If  H is  continuous  and  bounded,  then 

ip* 

lim  f H(x)du  (x)  =1  ( H(|°(t))dt  ' (2) 

e|0  JRn  e 1 JQ  x 

where  x is  any  point  on  r and  T*  is  the  period  of 

the  cycle,  £®(t)  = £®(t  + T*). 
x x 

We  prove  Theorem  in  Section  2 and  discuss  assumption  (A5) 
there.  From  Theorem  1 we  can  immediately  derive  a result  in 
partial  differential  equations  under  the  following  additional 
assumption. 

(A6)  There  exists  s’  > 0 such  that  for  0 < e < T there  is  a 
bounded  solution  ue  on  (t>  0)  XRn  to 

trace  (eaoTuxx)  + fux  - ut  = 0 (3) 

with  initial  condition  ue(0, x)  = H(x),  x e Rn.  The 
function  H is  assumed  continuous  and  bounded  as  in  Theorem  1. 

Assumption  (A6)  is  discussed  in  Section  2. 

The  solution  ue  to  (3)  satisfies  ue(t,x)  = EH(£e(t)).  Since 
(l)  has  for  each  e an  ergodic  measure,  then,  for  all  x 

lim  ue(t,x)  - f H(y)du  (y ) . (4) 

t->oo  ^n  e 


Hence  we  have  the  following 
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Theorem  2.  Assume  (Al)  - (A6).  Then 

<p* 

lim  lim  ue(t,x)  = — f H($°(t))dt  . * (5) 

e->0  t->co  1 J0  x 

However,  for  any  t > 0,  no  matter  how  large,  we  have 

lim  ue(t,x)  = H(£°(t))  . (6) 

e->0 

See  Fleming  [1],  Theorem  4.1.  Equations  (5)  and  (6)  show  that 
caution  must  be  observed  when  attempting  to  approximate  the  long 
term  behavior  of  a physical  system  subject  to  small  random 
disturbances . 

In  Section  3 we  consider  a class  of  problems  for  which  both 
the  invariant  measure  p and  the  limit  cycle  r can  be  computed 
explicitly.  We  evaluate  directly  both  sides  of  equation  (2). 

In  earlier  work  ([6], [7])  we  have  considered  equation  (l) 
when  the  corresponding  deterministic  dynamical  system  has  the 
origin  as  a globally  asymptotically  stable  equilibrium  point.  In 
Section  4 we  consider  that  case  and  prove  easily  the  following 

Theorem  3»  Assume  (Al),  (A5),  and 

(A21).  For  e = 0,  (1)  has  a unique  critical  point  at  x*  = 0 which 
is  globally  asymptotically  stable. 

Then,  if  B is  any  open  set  containing  the  origin 

lim  n (B)  = 1 . (7) 

e|0  e 


If  H is  continuous  and  bounded,  then 
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lim  / H(x )d(j.  (x)  = H(0)  . (8) 

e|0  JRn  e 

* 

Under  stronger  assumptions  and  using  a completely  different 
technique  we  established  in  [6]  an  expansion  of  the  left  hand  side 
of  (8)  in  powers  of  e.  One  might  attempt  to  derive  such  a result 
for  the  left  hand  side  of  (2).  We  give  an  example  to  illustrate 
the  potential  difficulty  in  deriving  such  an  expansion  without 
strong  assumptions. 


3 
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2.  In  this  section  we  first  prove  Theorem  1 and  then  discuss 
assumptions  (A5)  and  (A6). 

* 

Proof  of  Theorem  1.  We  first  establish  (Cl).  To  do  this  we  begin 
by  constructing  for  each  critical  point  x*  a neighborhood  N con- 
taining x*  such  that 

lim  p.  (N)  = 0 . (9) 

e->0  e 

For  notational  convenience  let  x*  = 0 denote  a critical  point. 

Since  assumption  (A3)  holds,  there  exists  a positive  definite 
matrix  P,  a positive  constant  c and  a neighborhood  Z of  x*  = 0 
such  that  if  V(x)  = x^Px,  then  V f > 0 in  Z (see  [5],  P*  296)  and 

X ■ 

T — 

also  trace  00  P _>  c in  Z.  Now  let  c-^,  Cg,  M be  positive  constants 
with  C-,  > Cp  > M and  such  that  the  set  S = (x  : V(x)  < c^  c Z. 

Define  Q = fx:  V(x)  < Cg],  N=(x:V(x)<  M],  and  let  0 < y < l/2 

be  such  that  d(SS,SQ)  > 2y  and  d(SN,SQ)  >27.  We  shall  need  these 
facts  below. 

We  claim  that  (9)  holds  for  the  above  N.  Suppose  not,  then 
we  shall  show  (in  a lengthy  argument)  that  this  leads  to  a contra- 
diction. First,  since  (9)  is  assumed  false,  there  exists  a 

positive  number  6 such  that  lim  u (N)  >5.  Using  assumptions 

e->0  e 

(A5),  (A3)  we  next  choose  R such  that  F c B(R),  all  critical 

points  lie  in  B(R-l),  and  ne(B(R)C)  < 6/6  for  0 < e < for  some 

fixed  £q  > 0. 

For  each  £ > 0,  let  £e(t)  be  the  solution  to  (l)  with  initial 
condition  distributed  according  to  the  measure  p.p.  Define  sets 
K®,  Kg,  K|,  Kj  by 
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t£  > §“  ?B(0)  e N}  , 
re  < — > ?£(0)  e N}  , 


K|  = (r(|f)  e N,  i£(0)  c B(R)  - N]  , 

K4  = G N>  £ B(R)C]  ' 

where  M is  the  maximum  of  V in  N and  t is  the  exit  time  of  £ (t) 
from  N if  |e(0)  e N. 

Since  p.  (N)  = P(£e(t)  e N]  for  any  t > 0,  then 

‘4 

a (N)  “ HI  P(K1E)  • U0) 

e fcl  1 

We  need  to  estimate  the  terms  on  the  right  side  of  the  previous 
equality.  First,  we  have  P{K^}  ^ 6/6. 

Let  us  estimate  PfK^'}  next.  If  x e N,  let  xx  be  the  exit 

time  of  £G(t)  from  N.  Then  using  the  It$  stochastic  differential 

rule  we  have  for  any  t > 0 that 


EV(£j(t  ATG))  - EV(x) 
rtATx 

> E j (vxf+e  trace  PaaT)Uj(t))dt  > ecE(  t a tg)  . 

0 

Then  M > ecE(tAT£)  where  M has  been  previously  defined  as  the 
maximum  of  V in  N.  Therefore  for  all  q > 0 


QP[t£ 


q] 


> 
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2M  e 2M  1 

Choosing  q = — , one  obtains  P[t  > — } < -!5.  From  this  fact  we 
have  that  P[K^}  m.£(N)/2. 

Let  us  now  estimate  P(K^).  To  do  this  we  first  derive  the 
estimate  (ll)  below.  let  T’  be  chosen  so  that  if  x e S-N,  then 
£®(t)  e Sc  for  t > T' . Note  that  once  the  trajectory  ^("k) 
leaves  N(Q, S)  it  never  returns  to  N(Q,  S).  Using  (A4),  let  R'  be 
chosen  so  that 

(4°(t)  : x e B(R)  , t > 0}  c B(R'-l) 
and  R"  be  chosen  so  that 

U?(t):  x e B(R'),  t > 0}  c B(R"-l)  . 

X v 

Now  choose  T > T'  so  that  if  x e B(R* ) - B(R),  then  £°(t)  e B(R) 
for  t > T*  and  d(£^(t),  c>B(R) ) > 2-y  for  t > T’ . This  can  be  dene 
using  (A4)  and  the  fact  that  B(R')-B(R-1)  contains  no  critical 
points.  Now  let  x be  any  random  initial  condition  such  that 
x e B(R' ) wpl.  Then  standard  estimates  yield  that 

pf  sup  U®(t)  -$2(t)|  > < 2n  exp  (-ye'l/2A)  (ll) 

l0<t<T 

where  n is  the  dimension  of  the  space  and  A is  a constant  depending 
upon  T and  a bound  on  a,  f . on  B(R").  See  [1],  Lemma  2.1  for  a 

X 

derivation  of  (ll). 

2M 

Now  for  sufficiently  small  e so  that  T < — , 
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P(K|}  = PUe(|f)  e N,  |e(0)  e B(R)  - N) 


<1 


(Se(|f)  e N,  ?e(jT)  ^ B(R')  -Q,  ") 


Pi 


y=i 


£e(kT)  € B(R')-Q,  k = 1,  ...,J-1. 


/ 


+ P[?£(~)  e N,  ?e(kT)  e B(R'  ) - Q,  k = 


Using  estimate  (11)  appropriately  we  have  that 

P{K|)  < (|J^  + l)2n  exp  (-ye”1/2*)  . 


Note  that  P(K^)  -+  0 as  ei  0. 

It  remains  to  estimate  P(k|).  The  construction  of  the 
estimate  for  P(Kg)  is  similar  to  that  for  P(k|)  except  that  we 
condition  upon  the  first  time  that  i (t)  exits  from  N.  One  obtains 
that  P(k|]  — 0 as  e | 0. 

Hence  we  have 


6 = lim  n (N)  <|  + 0+0  + 
e-1-0  e 


6 _ 46 
5 ~ IT  * 


However,  this  is  impossible  for  6 > 0.  Thus  6 must  be  0 and 
therefore  we  can  construct  a neighborhood  of  the  critical  point 
such  that  (9)  holds. 

Now  suppose  there  are  k critical  points  x^,  ...,xk  and  let 

Ni  be  a neighborhood  of  x^  constructed  by  the  above  procedure. 

Now  take  any  compact  set  B not  intersecting  the  cycle  r nor 

k 

l J N,  . We  claim  lim  a (B)  = 0.  Suppose  not,  then  lim  p.  (b)  = 6 

i=l  1 eiO  e elO  e 

for  some  6 > 0.  Choose  R so  that  B c B(R),  each  N^  c B(R)  and 
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such  that  |i  (B(R)  ) < 6/6.  Let  cjd  be  such  that  d(T,B)  > 2 a).  Then 

k 

choose  T such  that  if  x e B(R)  - M N. , then  d(£  (t),T)  < to  for 

i=l  1 

t _>  T.  Using  the  estimate  (11)  again  we  have  that 

P i)StPT  >°}-o  as  0 * (12i 


Hence 


H£(B)  = P[eS(T)  e B] 

.ki 

= P[^e(T)  e B,  £e  ( 0)  e B(R)  - Q 

i=l  1 

+ P[?e(T)  e B,  l£(0)  e Q N . ] 

i=l  1 

+ P[?£(T)  e B,  ?e(0)  c B(R)C]  . 


Notice  that  the  third  term  on  the  right  side  is  bounded  above  by 
6/6  and  the  second  term  by  ) tas(N. ) which  has  limit  0 as  e | 0. 

Using  estimate  (12)  we  have  that  the  first  term  has  limit  zero  as 
e | 0.  Hence 

6 = lim  u (B)  < 6/6 

ejo 

contradicting  the  fact  that  6 > 0.  Therefore  6 = 0 and  lim  (j.  (B)=  0. 

ejo 

We  are  now  ready  to  establish  (Cl).  Let  B be  any  open  set 
containing  r.  For  any  R such  that  B <z  B(R), 

1 = n£(B)  +he(B(R)  -B)  +h£(B(R)C) 

< (B)  +de(B(R)  -B)  + 6 


where  6 is  a bound  of  |i  (B(R)C)  for  0 < e < e«.  Hence 

e 0 
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where  y is  any  point  on  the  limit  cycle  r.  This  completes  the 

proof  of  Theorem  1.  Q.E.D. 

Theorem  2 follows  immediately  from  Theorem  1 as  discussed 

in  the  introduction.  Let  us  now  discuss  first  assumption  (A5) 

and  then  (A6).  Khasminski  [8],  Wonham  [10],  Zakai  [11],  and 

Kushner  [9]  have  given  criteria  for  the  existence  of  an  ergodic 

T 

measure  p.£.  The  results  of  [8]  and  [9]  depend  upon  aa  being 
uniformly  positive  definite  while  the  others  do  not.  The  remaining 
condition  in  (A5)  depends  upon  the  construction  of  an  appropriate 
Liapunov  function,  see  [9]  or  [10].  A class  of  problems  for  which 
(A5)  is  satisfied  is  the  following: 


r ^ 

12 

(Z)  f(x)  = Ax+og(x),  a is  constant,  g and  Vg  are  bounded  on  Rn, 

A has  eigenvalues  with  negative  real  parts,  (A, B)  is 
controllable  and  satisfies  condition  (CO).  ' 

For  a proof  of  this  fact  see  the  Examples  on  p.  227  and  Corollary  6 
on  p.  228  in  [9] • The  definition  of  condition  (CO),  a mild 
restriction  in  normal  applications,  is  given  there.  Controlla- 
bility means  that  rank  (B, AB, . . ., An-1B)  = n. 

We  now  discuss  (A 6).  If  era  is_  uniformly  positive  definite, 
then  conditions  guaranteeing  the  existence  of  a smooth  solution 

can  be  fouhd  in  Theorem  12,  p.  25  in  [2].  However,  it  is  not 

T 

necessary  that  aa  be  positive  definite  to  have  a smooth  solution, 
see  Theorem  1,  p.  73  in  [4] . 


13 


3.  In  some  special  cases  the  invariant  measure  can  he  computed 
explicitly.  The  left  hand  side  of  (2)  can  then  he  evaluated 
using  Laplace's  method.  We  illustrate  that  technique.  Let  V(x) 
he  a smooth  function  such  that 


(-V(x)/e)dx  < oo 


and  let  h(x)  = (h1(x), . . .,hn(x) ) satisfy  the  equations 


Vh*VV  = 0 , * 
div  h = 0 . 

Then  the  unique  invariant  probability  density  associated  with  the 
stochastic  differential  equation 

d£  - -W(£)  + h(£)dt  + (2e)1//2dw(t)  (13) 

is  pe(x)  = C-jJe)-1  exp  (-V(x)/e).  To  prove  this  one  simply  checks 
that  pe  satisfies  the  Fokker- Planck  equation. 

Let  x e R2,  V(x)  = (l-x2-x2)2  , h^x)  = 4x^  and  h2(x)=  -2x1. 

2 4 

Then  r = (x  :x1+x2  = 1}  is  a stable  limit  cycle  whose  domain  of 

2 

attraction  is  R - (0). 


2 4 

Making  the  change  of  variables  s = x1+x2»  t = x^,  and  using 
symmetry,  one  obtains  that 
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D(e)  = 


cr»  /i 

J j R1(s,t)  exp  (-(l_s)2/2E)dtd£ 
0 0 


CO 


J J exP  (-(1-s)2/2e)dtds 


0 0 


where  R1(s,t)  = 4_1(  s-t2  )-1/^  and  R2(s,t)  = 4-1(s-t2)"’^//^'. 


s /s 

Define  M-L(s)  = J R1(s,t)dt  and  M2(s)  = J R2(s,t)dt.  Utilizing 

0 0 
Laplace's  method  one  obtains  that 


lim  D( s ) = 


M1(l) 


n I (1-t2) 


-1/4 


dt 


:|0 


cm " ~ T 


(14) 


f U-t2) 

u 


-3/4 


dt 


0 


Now  let  £^(t),  ^2(t)  be  the  solution  to  (lj)  with  e = 0 and 

p 2i 

^(0)  = 0,  |2(0)  = 1.  Since  £^(t)+£2(t)  = 1,  then  the  right  hand 
side  of  (2)  is 

T*/4  ______ 

f / 1 “ ^i(4)  dt  * 


0 


Making  the  change  of  variables  s = £^(t),  one  again  obtains  (l4). 
The  conclusion  of  Theorem  1 holds  for  this  example  even  though 
assumption  (A2)  is  not  satisfied.  Note  that  the  matrix  f (0)  has 
0,  2 as  eigenvalues. 


1 
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4.  In  this  section  we  consider  a new  case  where  the  origin  is 
the  unique  critical  point  of  (1)  with  e = 0 and  is  globally 
asymptotically  stable.  It  is  then  easy  to  establish  Theorem  3 
using  a method  similar  to  that  employed  in  the  proof  of  Theorem  1. 
The  details  are  omitted. 

However,  in  [6]  under  stronger  assumptions  than  in  Theorem  3 
we  established  an  expansion 


H(x)d^e(x) 


k 

rz 

j=l 


Cj£‘ 


+ o(e 


(15) 


valid  for  any  positive  integer  k.  The  constants  c . can  be  found 

J 

by  solving  linear  algebraic  equations  with  cn  = H(0).  Among  the 
• u 

assumptions  in  [6]  are  that  the  matrix  f (0)  have  only  eigenvalues 

A 

with  negative  real  parts.  See  [6]  or  [7]  for  details. 

The  expansion  (15)  Is  not  always  to  be  expected.  Consider 
3 1/2 

d£  = -£"at  + (2e)  ' dw(t).  This  equation  has  an  invariant  measure 
for  e > 0 given  by  B(e)  exp  (-x^/4e)  where  B(e)  is  a normalizing 
constant.  An  easy  calculation  shows  that 

J x2dn£(x)  = Ce1//2 

R 

for  an  appropriate  constant  C.  This  contradicts  the  expansion  (15) 
The  difficulty  is  that  fx(0)  = 0 so  that  the  rate  of  approach  to 
the  origin  is  too  slow. 

One  might  be  tempted  to  derive  an  expansion  of  (2)  in  powers 
of  e.  However,  one  will  probably  need  an  additional  assumption 
guaranteeing  that  the  rate  of  approach  of  the  deterministic  tra- 
jectories to  T near  r is  "sufficiently  fast". 
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